NATURAL LANGUAGE PRocEssiNG FOR INFORMATION RETRIEVAL AND KNOWLEDGE DISCOVERY

Natural Language Processing (NLP) is a powerful technology for the vital tasks of information retrieval (IR) and knowledge discovery (KD) which, in turn, feed the visualization systems of the present and future and enable knowledge workers to focus more of their time on the vital tasks of analysis and prediction. First, a definition of NLP. Natural language processing is a set of computational techniques for analyzing and representing naturally occurring texts at one or more levels of linguistic analysis for the purpose of achieving human-like language processing for a range of tasks or applications. The goal of researchers and developers of NLP is to produce systems that process text of any type, the same way which we, as humans, do-systems that take written or spoken text and extract what is meant at different levels at which meaning is conveyed in language. NLP is used for a wide range of tasks or applications. This discussion will focus on two particular tasks, namely information retrieval (IR) and knowledge discovery (KD). Figure l shows the levels of language processing at which cognitive linguists hypothesize that humans understand or extract meaning. An interesting point to note is that, while meaning is frequently thought to be conveyed at the level of language represented as "semantics," the fo l-lowing explanation will clarify how meaning is in fact conveyed and how we, as humans, extract meaning at every level of language, not just at the semantic level.